What is claimed is: 

1. A method for distinguishing between background and foreground objects in an image, 
comprising the steps of: 

i) determining respective range values for pixels in each of a plurality of related images; 

ii) determining respective intensity values for pixels in each of said plurality of related 
images; 

iii) establishing a multi-dimensional background model for at least some of said pixels on 
the basis of said range values and said intensity values; 

iv) comparing the range value for a particular pixel in a given image with the background 
model for that pixel, and labeling the pixel as a foreground pixel if the range value differs from 
the background model by a range threshold; and 

v) comparing the intensity value for said particular pixel in a given image with the 
background model for that pixel, and labeling the pixel as a foreground pixel if the intensity 
value differs from the background model by an intensity threshold. 

2. The method of claim 1 further including the step of determining whether the range value for 
said particular pixel is valid, and performing said comparing step (iv) only if said range value is 
valid. 

3. The method of claim 2 wherein determining whether the range value for said particular pixel 
is valid further includes: 

computing a confidence value; and 

comparing said confidence value to a threshold to determine if the range value for said 
particular pixel is valid. 
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4. The method of claim 2 further including the step of determining whether background range 
data for said particular pixel is valid, and if said background range data is not valid, labeling the 
particular pixel as a foreground pixel in response to the comparing step (iv) for neighboring 
pixels if a range gradient associated with said particular pixel is less than a gradient threshold. 

5. The method of claim 1 further including the step of determining whether the luminance of 
said particular pixel is greater than a designated minimum value, and performing said comparing 
step (v) only if the luminance is greater than said minimum value . 

6. The method of claim 1 further including the step of determining the ratio of the intensity 
value for said particular pixel to the background intensity value for that pixel, and labeling the 
pixel as a foreground pixel if said ratio is less than a first predetermined value. 

7. The method of claim 6 wherein said first predetermined value is based upon a decrease in 
luminance due to shadows in an image. 

8. The method of claim 6 further including the step of labeling the pixel as a foreground pixel if 
said ratio is greater than a second predetermined value. 

9. The method of claim 8 wherein said second predetermined value is based upon an increase in 
luminance due to interreflections in an image. 

10. The method of claim 1 further including the step of alternatively labeling the pixel as a 
foreground pixel if the luminance of the pixel is greater than a designated minimum value by a 
predetermined factor. 

11. The method of claim 10 wherein said factor is at least 2. 

12. The method of claim 10 wherein said alternative labeling step is performed only if the 
luminance of the background texture model for the pixel is less than said designated minimum 
value. 
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13. The method of claim 1 further including the step of updating said background model on the 
basis of range values and intensity values obtained from a sequence of N images. 

14. The method of claim 13 further including the steps of detecting image-to-image changes in 
said range values and intensity values, and effectively adjusting the value of N in accordance 
with the detected changes. 

15. The method of claim 14 wherein the value of N is effectively increased when the magnitude 
of detected changes increases. 

16. The method of claim 1 wherein said intensity values are one-dimensional values 
corresponding to the luminance of pixels. 

17. The method of claim 1 wherein said intensity values are two-dimensional values which are 
invariant to luminance. 

18. The method of claim 1 wherein said intensity values represent a three-dimensional color 
space. 

19. The method of claim 18 further including the step of normalizing said intensity values 
according to luminance. 
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20. A system for identifying foreground objects in an image comprising: 

a sensor which determines intensity values for pixels within an image; 

a range processor which determines distance values for pixels within said image; 

a background estimator which computes a multi-dimensional background model for said 
pixels on the basis of said intensity values and said distance values; and 

a discriminator which compares intensity and distance values for pixels in an image to 
said background model and determines whether said pixels represent a foreground object. 

21. The system of claim 20 wherein said background estimator generates a multi-dimensional 
histogram of intensity and distance values for a pixel within a plurality of related images, and 
computes a background model for said pixel by means of a clustering technique. 

22. The system of claim 20 wherein said discriminator determines the differences between the 
intensity and distance values for a pixel in an image and the background model for that pixel, and 
compares said differences to respective intensity and distance thresholds. 

23. The system of claim 22 wherein said discriminator selectively adjusts said intensity 
threshold in accordance with the result of a comparison based upon said distance threshold. 

24. The system of claim 22 wherein said discriminator increases said intensity threshold if the 
difference between the distance value for a pixel in an image and the background model for that 
pixel is less than said distance threshold. 

25. The system of claim 20 wherein said intensity values are one-dimensional values 
corresponding to the luminance of pixels. 

26. The system of claim 20 wherein said intensity values are two-dimensional values which are 
invariant to luminance. 
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27. The system of claim 20 wherein said intensity values represent a three-dimensional color 
space. 

28. The system of claim 20 further including an intensity processor which normalizes said 
intensity values according to luminance. 

29. A method for determining a background model in a scene, comprising: 

estimating a background model using a multi-dimensional histogram for a sequence of 

frames; 

recording pixel values in said multi-dimensional histogram; 

configuring said pixel values into a cluster; and 

selecting a largest cluster as representing said background model. 

30. A method for determining a background model in a scene as recited in claim 34 wherein 
selecting said cluster as said background model further comprises selecting a cluster having a 
deepest range value over said sequence of frames. 

3 1 . A method for determining a background model in a scene as recited in claim 34 wherein 
selecting said largest cluster as representing said background model further comprises selecting a 
static background model. 

32. A method for determining a background model in a scene as recited in claim 34 wherein 
selecting said largest cluster as representing said background model further comprises 
selecting a dynamic background model. 

33. A system for determining a background model in a scene comprising: 

a background estimator configured to estimate a background model using a multi- 
dimensional histogram for a sequence of frames; and 

a processor configured to record pixel values in said multi-dimensional histogram, 
configure said pixel values into a cluster, and selecting a largest cluster as representing said 
background model. 
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